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Abstract 

This study investigated using a state’s Part C early intervention accountability tool to 
increase the number of outcomes meeting compliance within IFSPs. The Case Review 
Tool (CRT) was used to examine differences from year one to year three on three 
measures of quality outcomes. There was no evidence of change in two of the measures, 
but there was evidence that IFSP outcomes increased in the components of functional and 
measureable after three years of using the CRT. There was also an increase in the 
number of outcomes from year one to year three. 

Using an Accountability Tool to Improve the Quality of Outcomes 
on Individual Family Service Plans 

Federal programs require monitoring and accountability systems at the state level to 
assure that early intervention services are delivered to families and their child with 
disabilities according to standards outlined in the Individuals with Disabilities Education 
Improvement Act (IDEA, 2004). Accountability serves an important purpose in early 
intervention programs under Part C of IDEA, the birth to three early intervention systems 
for infants and toddlers with developmental delays and their families. Every child 
eligible for Part C services must have an Individualized Family Service Plan (IFSP) 
based on their strengths and challenges, including outcomes outlining the goals of the 
upcoming year. Early interventionists, those who work in the birth to three Part C 
programs, work with families to write outcomes based on the needs of the child and 
family. 

The assessment of functional outcomes embedded in routines is often difficult (Bradley et 
ah, 2007; Jung & Baird, 2003) due to the qualitative aspect of the process. Early 
interventionists continue to struggle to create quality outcomes on a consistent basis 
(NECTAC, 2008). The development of outcomes for the IFSP is “complicated by the 
fact that most professionals in early intervention programs have little training in assessing 
family needs,” which is vital to the determination of meaningful outcomes (Bailey & 
Simeonsson, 2001, p. 117). 

The outcomes are the foundation for services within the IFSP and serve a vital role in 
meeting the needs of the child. According to IDEA 2004, IFSP’s must include 
measurable results and be developmentally appropriate. In order to this outcomes must 
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be embedded in family routines, but this is challenging. The early interventionist’s role is 
to assist the team in formalizing outcomes, but teams continue to need performance 
support in order to meet compliance standards set forth by the federal government to 
write meaningful, quality outcomes. Despite single professional development training 
events, early interventionists continually struggle to write quality IFSP’s with functional 
and measurable outcomes (Jung, 2010; McWilliam, 2010). 

The purpose of this study is to determine if a tool used for state accountability purpose 
can increase the quality of IFSP outcomes for early interventionists in the field. One 
state’s monitoring process uses a quantitative tool to merge professional development, 
technical assistance, and accountability systems. This targets increased adherence to 
federal regulations and quality outcomes within IFSP’s over time. 

Literature Review 


State Models 

Over the years, state systems have used varied approaches to increase the overall quality 
of IFSP as well as comply with federal regulations. The National Early Childhood 
Technical Assistance Center (NECTAC) provides resources and guidance to state 
systems to create improved early intervention and accountability programs. For example, 
in 2007, Wyoming worked with NECTAC to improve its Part C monitoring and general 
supervision system. The system included, “multiple methods to: ensure implementation 
of IDEA and the accountability of regional programs and their providers; identify and 
correct noncompliance; facilitate improvement; and support practices that improve results 
and functional outcomes” (Kasprazak, Hurth, Lucas, Marshall, Terrell & Jones, 2010). 
The result was a new monitoring manual and a procedure manual to help staff understand 
new state procedures (Kasprazak, et ah, 2010). 

The state of Missouri also worked with NECTAC to improve Part C services. The focus 
of the technical assistance from NECTAC was to build family capacity in everyday 
routines and activities. A plan was made to “include a component to increase the 
knowledge and skills of local programs and practitioners so that they were better able to 
develop high quality IFSPs in strengthened partnership with families” (Kasprazak, et al., 
2010). As a result, Missouri First Steps IFSP Quality Indicator Rating Scale (QIRS) was 
developed in 2005 to train staff about quality indicators in IFSPs and the monitoring 
process. The new system using the QIRS created a monitoring system with built-in 
training materials for staff and real-time data sharing with the ultimate goal of improving 
the quality of IFSPs and family centered services (Kasprazak, et al., 2010). 

Many states rely on IFSP guidance documents to give support to early intervention teams 
writing IFSP’s in the field. The National Early Childhood Technical Assistance Center 
(NECTAC, 2008) links together many states’ IFSP forms along with supplemental 
documents on their website for states to access. Some of the IFSPs also include prompts 
within the document to help teams better understand what information should be included 
in each IFSP section. 

Improving IFSP Quality 
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There have been few studies that have addressed the development of quality IFSPs. One 
of these studies addressed the use of IFSP prompts to increase the overall quality of 
IFSP’s. Lee Ann Jung’s 2010 study compared IFSP’s written before and six months after 
the IFSP fonn was revised with added instructions, or prompts. The prompts were added 
to aid staff to know what needed to be included in each section of the IFSP. The staff 
received an IFSP manual with a written narrative and guidance in completing the form. 
The form’s outcomes section was revised to include “prompts helping staff to connect 
routines, priorities, and outcomes” (Jung, 2010, p. 205). The IFSP prompts increased the 
family-centered language, but the “outcomes written using the revised form were not 
measurable and remained far below criteria” (Jung, 2010, p. 210). The outcomes 
improved in their relation to family routines, but their measurability continued to lack 
progress. 

In 2003, Jung & Baird completed a study on IFSP quality. The study reviewed 120 
IFSP’s and rated the quality of nine indicators using the IFSP Rating Scale (McWilliam 
& Jung, 2001). Some of the IFSP’s in the study were submitted by service coordinators 
who had attended a three-day training on IFSP development. Jung & Baird note that 
“although the training might be useful in improving IFSP writing skills, it is not sufficient 
to result in adequate competency.” The three-day training was a much more costly event 
than the inclusion of IFSP prompts with much the same results. 

Performance Support 

Studies (Dunst & Raab, 2010; Trivette, Dunst, Hamby & O’Herin, 2009; Dunst & 
Trivette, 2009) indicate that learning opportunities for professionals that include 
components of self-assessment, active involvement, and learning over time create 
effective learning. The best gains in training happen when the professional learner is 
actively engaged in the process. Also, “the more the learner is engaged in reflection on 
those opportunities using some external set of standards, the greater the likelihood of 
optimal benefits” (Dunst et al, 2009, p. 11). Early intervention training opportunities 
need to be mindful of integrating an evidence-based approach to professional 
development. Using this information, accountability and professional development could 
be merged to create an active, ongoing process that integrates with a state’s 
accountability system. 

Case Review Tool 

In a mid-western state, the accountability system used a Case Review Tool (CRT) to rate 
compliance measures in IFSP’s. The CRT was used to increase overall IFSP quality, 
including the quality of functional and measurable outcomes, although the CRT’s main 
purpose is for yearly monitoring and compliance checks in all of the state’s regional early 
intervention programs. Individual regions in the state also used the CRT as a training 
method with their staff to review the varied components of compliance. The state 
required all regions to use the CRT once a year for compliance accountability. Within 
each region, local early intervention staffs were internal reviewers. External reviewers 
also reviewed IFSP’s for compliance using the CRT. The state Part C office provided 
feedback to the regions with both sets of data. 
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All regional staff had access to the CRT, which provides guidance in writing the IFSP 
through a narrative rubric. The CRT was also used for yearly accountability purposes. 
This makes the CRT a tool that integrated the professional development, accountability, 
and technical assistance systems of the state to improve quality outcomes over time. 

Purpose of Study 

The purpose of this study was to determine if using an accountability measurement tool 
over time increased the number of outcomes meeting compliance within IFSP’s. 
Specifically, did the use of this tool make a difference from year one to year three in IFSP 
outcomes that (a) correlate with family priorities and concerns, (b) are functional, 
measurable, and related to everyday routines, and (c) are developmentally appropriate? 

Method 

Participants 

This was a retrospective study reviewing data extracted from the mid-western state’s 
accountability tool. In total, 40 charts were chosen randomly from the early intervention 
system case review process for yearly accountability monitoring. A comparison of 20 
charts from 2007, year one of implementation of the accountability tool, and 20 charts 
from 2010, year three of the process, was completed. 

Procedures 

Instrument. The instrument is the Case Review Tool (CRT) which contains a 4-point 
rating scale with ratings of 1-unacceptable, 2-minimally acceptable, 3-practice standard, 
and 4-best practice. The CRT was adapted from the Missouri First Steps IFSP Quality 
Indicator Rating Scale (2005). There were three measures from the CRT included in this 
study. The first measure, Measure A, was child and family outcomes correlate with 
family priorities and concerns relative to the child’s development. The second measure, 
Measure B, was child outcomes are functional, measureable (including criteria, 
procedures, and timelines) and related to participation in everyday routines. The third 
measure, Measure C, was child outcomes are developmentally appropriate and can 
realistically be achieved in the given review period. A summary of each of the three 
measures as well as a description of a 1, 2, 3, and 4 rating are provided in Figure 1 
(Figure 1 was unable to be uploaded online—Please contact the author for image). 

This rating was used on each outcome included on the IFSP. For example, if an IFSP had 
five outcomes, items from the three measurements were used for each of the five 
outcomes. Furthermore, the second measure includes a rating on two individual 
components (functional and measurable). Functional was defined as related to 
participation in every day routines and activities. Measurable was defined as including 
criteria, procedures, and timelines. Using the comments section of the tool, data was 
collected identifying whether each of the components of functional and measurable were 
contained in each outcome on the IFSP. 
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Rating procedures. Use of the CRT was implemented in 2007 in a mid-westem state. 

The CRT is used by external and internal evaluators within the state. The external 
evaluators consisted of contracted technical assistance team made up of a minimum of 
three expert early intervention consultants. The internal evaluators were at the program 
level. These internal evaluators were made up of a team of a minimum of three early 
interventionists in the following roles: early intervention program staff, service 
coordinator, and experienced parent (regionally hired parent working with the early 
intervention program). 

Each year, a random number of charts, depending on the size of the program, are 
sampled across the state. The external and internal evaluation teams each review one 
third of the sample. The final third of the sample is reviewed by both sets of evaluators 
to establish inter-rater reliability. Each team submits the CRT ratings electronically to 
the state Part C office. The results of the rates are compiled by the Part C data officer 
into a regional report. 

Data Collection 

The outcome measure ratings and comments from the three identified measures were 
collected from 20 random charts from year one and 20 random charts from year three. 

The rating measurements of one, two, three, or four, were placed into an excel 
spreadsheet for further analysis. The comments on each outcome were reviewed to 
determine whether it contained functional and measurable components. If the component 
was present, it was coded as a 1, and if it was not present, it was coded as a 0. 

Data Analysis 

Descriptive statistics were generated for year one and year three of the CRT items. An 
univariate analyses of variate (ANOVA) was used to investigate the change in individual 
items between groups year one and year three. In this analysis, the three measures from 
the CRT were used to compare year one to year three. Further analysis was completed to 
understand the impact of the components of functional and measurable on the CRT 
measures, and the relationship between the number of outcomes in year one and year 
three was examined. All analyses were conducted using PASW 18.0. 

Results 

Descriptive statistics for each outcome are presented in Table 2 for the components of 
measurable and functional based on a zero or one scale (zero, not met and one, met) 
including means and standard deviations for year one and year three. Table 3 presents 
the descriptive statistics for the individual measures from the CRT of (a) family concern, 
(b) functional/measurable, and (c) developmentally appropriate, based on the 1-4 scale 
used in the CRT. 

Univariate results are presented in Table 4 and 5. Table 4 showed no significance 
between the CRT measures of family concerns, F (1,38) = .060,/? = .807, and 
developmentally appropriate, F (1,38) = .069,/? = .794 from year one to year three. 
Significance was indicated for the measure of functional/measurable, F (1,38) = 4.77,/? = 
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.035, but this was a decrease from year one to year three with use of the CRT. There was 
a significant difference from year one to year three in the number of outcomes, F (1,38) = 
11.37,/? = .002, indicating that more outcomes were written per IFSP in year three than 
year one. 

The functional/measurable item on the CRT includes both functional and measurable 
components in the score for the rating, and the rating encompasses all of the outcomes for 
the specific IFSP being rated. Since there was significance in this exploratory study, the 
components were analyzed individually to determine the impact of each component on 
the rating, and to analyze each individual outcome. The data from year three on Table 5 
showed significance for both functional and measurable components of outcomes with 
measurable, F ( 1,228) = 10.287,/? = .002, and functional, F (1,228) = 19.124,/? = .000. 
More data was available in year three due to the increased number of outcomes on 
IFSP’s. 

Since there was significance for the number of outcomes increasing from year one to year 
three, a follow-up analysis of linear regression was completed for the number of 
outcomes (Table 6) in comparison to the average score. Even though this linear model 
proved to be a poor fit when looking at Figure 1 for estimated marginal means, there was 
a slightly higher increase for scores for IFSP’s with five or more outcomes. It appeared 
that five or more outcomes indicated higher scores on the CRT than four outcomes or 
less. Therefore, a univariate test was completed for average score (Table 7) of the three 
measures on the CRT with the number of outcomes, F (1,39) = 5.198,/? = .028. The 
results were significant, in that there was an increase in average IFSP score when there 
were five or more outcomes versus four or fewer outcomes. 

Table 2 

Descriptive Statistics of Measurable and Functional Components for Each Outcome 




Year 1 



Year 3 



M 

SD 

N 

M 

SD 

N 

Measurable 

.54 

.50 

94 

.74 

.44 

136 

Functional 

.37 

.49 

94 

.65 

.48 

136 


Table 3 

Descriptive Statistics for Measures of Family Concerns, Functional/Measurable, and 

Developmentally Appropriate Outcomes 




Year 1 



Year 3 



M 

SD 

N 

M 

SD 

N 

Family Concerns 

2.80 

.62 

20 

2.85 

.67 

20 

Functional/ 

2.45 

.76 

20 

1.95 

.69 

20 

Measurable 

Developmentally 

2.80 

.62 

20 

2.85 

.587 

20 
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Appropriate 

Number of 4.70 1.78 20 6.80 2.14 20 

Outcomes 


Table 4 

Univariate Results for Effects of the Family Concerns, Functional/Measurable, and 


Develop mentally Appropriate Measures from the Case Review Tool (CRT) 




Sum of 

Df 

Mean 

F 

Sig 



Squares 


Square 



Family Concerns 

Between 

.025 

1 

.02 

.060 

.807 


Groups 

Within 

15.75 

38 

.41 




Groups 







Total 

15.77 

39 




Functional/ 

Between 

2.50 

1 

2.50 

4.77 

.035 

Measurable 

Groups 

Within 

19.90 

38 

.52 




Groups 







Total 

22.40 

39 




Developmentally 

Between 

.025 

1 

.025 

.069 

.794 

Appropriate 

Groups 

Within 

13.75 

38 

.362 




Groups 







Total 

13.77 

39 




Number of 

Between 

44.10 

1 

44.10 

11.37 

.002 

Outcomes 

Groups 

Within 

147.40 

38 

3.88 




Groups 







Total 

191.50 

39 





Table 5 

Univariate Results for the Effects of the Measurable and Functional Components on 


Individualized Family Service Plan (IFSP) Outcomes 




Sum of 

Df 

Mean 

F 

Sig 

Measurable 

Between 

Squares 

2.225 

1 

Square 

2.225 

10.287 

.002 


Groups 

Within 

49.322 

228 

.216 




Groups 

Total 

51.548 

229 




Functional 

Between 

4.422 

1 

4.422 

19.124 

.000 


Groups 

Within 

52.725 

228 

.231 
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Groups 

Total 57.148 229 


Table 6 

Linear Model of Number of Outcomes vs. Average score of the Family Concerns, 
Functional/Measurable, and Developmentally Appropriate Measures from the Case 

Review Tool (CRT) 


Source 

Type III 

Sum of 
Squares 

Df 

Mean Square 

F 

Sig 

Corrected 

1.814 a 

8 

.227 

.846 

.571 

Model 

Intercept 

207.045 

1 

207.045 

772.503 

.000 

Outcome 

Number 

1.814 

8 

.227 

.846 

.571 

Error 

8.309 

31 

.268 



Total 

284.00 

40 




Corrected 

10.122 

39 





Total 

a. R Squared =.179 (Adjusted R Squared = -.033) 


Table 7 


Univariate results of Estimated Marginal Means for Average Scores of the Three 
Case Review Tool (CRT) Measures of Small, less than four, and Large, five or more, 

Cases. 


Source 

Type III 

Sum of 
Squares 

Df 

Mean Square 

F 

Sig 

Corrected 

1.218a 

1 

1.218 

5.198 

.028 

Model 

Intercept 

203.985 

1 

203.985 

870.534 

.000 

Outcome 

Number 

1.218 

1 

1.218 

5.198 

.028 

Error 

8.904 

38 

.234 



Total 

284.00 

40 




Corrected 

10.122 

39 




Total 







a. R Squared = . 120 (Adjusted R Squared = -.097) 


Figure 1 
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Estimated Marginal Means for Average Score of the Three 
Case Review Tool (CRT) Measures 



Number of Outcomes 


Discussion 

Results of this study indicate that using a state accountability system did not show 
evidence of improving Measures A or C. Significance was found in the measure (b) child 
outcomes that are functional, measurable, and related to everyday routines, but this 
indicated a decrease in the rating on the CRT. 

IFSP Outcome Components 

It is important to note that there was more data in year three (N=136) than in year one 
(N=94) as seen in Table 2. This was due to an increase in the number of outcomes in 
IFSP’s in year three. In year one, the outcomes were very broad and general. As staff 
skill increased in year three, the outcomes were more specific, functional, and tied to 
everyday routines, thus requiring additional outcomes. 

This study found that the individual components of functional and measurable increased 
with use of an accountability tool. This may be due to the North Dakota early 
intervention system’s use of internal evaluators at the program level using the CRT tool. 
Since the internal evaluators are early intervention staff, it is more likely that the CRT is 
used for guidance when writing IFSP’s as a team. In addition to this, the state Part C 
staff promotes the use of the CRT in training and supporting staff performance. 

Number of Outcomes 

Earlier studies (Jung & Baird, 2003; McWilliam, Ferguson, et ah, 1998; and McWilliams 
2010) report that early interventionists need additional help in writing meaningful 
outcomes. McWilliams (2010) reports that measurability and quality are compromised 
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when there are fewer than three outcomes, and the outcomes written tend to be more 
general. When there are fewer outcomes, it seems that the outcomes may be less specific 
with little meaning to the family. Broad, general outcomes also provide less information 
useful for programming. 

The findings of this study, as discussed earlier, show that prior to using an accountability 
tool in year one, early interventionists had developed fewer outcomes. Over time with 
the use of the accountability tool, the IFSP’s were written with more functional and 
measurable outcomes related to everyday activities. Figure 5 demonstrates that when 
IFSP’s had five or more outcomes, they received higher ratings on the CRT. When early 
interventionists write outcomes related to participation in everyday family activities, 
more outcomes are necessary. When outcomes are meaningful, the number of outcomes 
increase as necessary to meet the family’s needs. The increase in the number of 
outcomes may also provide more specific guidance for programming to the early 
intervention team. 


Conclusion 

Following the three year use of an accountability tool describing the specific criteria to 
meet compliance with federal and state regulations, IFSP’s in this study included more 
outcomes that met the criteria for (b) child outcomes that are functional, measurable, and 
related to everyday routines. No change was noted in (a) outcomes that correlate with 
family priorities and concerns and (c) outcomes that are developmentally appropriate. 

One finding that came out of this study is that when five or more outcomes were included 
on an IFSP, the measurable and functional components increased. This needs further 
study to determine why this happened and how to support teams to create outcomes that 
are not so general they could appear on any IFSP. 

Implications for Practice 

This study highlighted the need to review all measures of the CRT to examine the 
connection between the measures and meeting compliance. The focus should be on 
further investigating whether using an accountability measurement tool over time 
increases early interventionists’ ability to write IFSP’s that meet state and federal 
compliance measures. In addition to this, it will be important to determine is if this 
makes a difference in the quality of services the family receives. 

Family assessment is an important step in creating specific, meaningful outcomes. Each 
of the three measures studied were related to information the IFSP team gathers from 
family assessment. Since the state in this study has recently emphasized incorporating 
the use of a formalized family assessment, the Routines-Based Interview (McWilliams, 
2009), there may be opportunities to look at the link between these measures and family 
assessment in the future. 
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Previous studies focused on intensive in-person training and IFSP prompts to promote 
quality IFSP’s, including well-written, functional, measurable outcomes. The key to 
improving IFSP quality may be in using a state’s available accountability and monitoring 
structures to train staff in a connected system that can be used for broader purposes, such 
as also increasing the quality of the IFSP and the outcomes within it. Jung suggests that 
although the “revision of the IFSP fonn may improve quality, it is not a substitute for 
high-quality professional development and technical assistance” (2010, p. 207). Thus, 
one method may not be the answer to improving the quality of IFSP’s, but a well- 
coordinated effort including many of the state’s resources might lead to better results. 

Research in professional development gives us evidence-based information about how to 
effectively train early intervention professionals. The CRT integrates accountability and 
professional development into an active, ongoing process over time. Using the CRT in 
daily work makes it a functional, dynamic tool that aids interventionists in continually 
framing what is needed to write a quality IFSP. The CRT is used for monitoring purposes 
on a yearly basis, and regional teams are actively involved in the process. The ongoing 
use of the CRT in the field and for accountability builds relevance to staff at the local 
level. 

Perhaps state systems need to find what Dunst & Trivette (2009) refer to as a “middle 
ground” where accountability is merged with training and technical assistance 
experiences. Accountability should be a part of the continuum of professional 
development to increase quality of the IFSP and service delivery to families. 
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